Abstract
In natural outdoor settings, advanced perception systems and learning strategies are major requirement for an autonomous vehicle to sense and understand the surrounding environment, recognizing artificial and natural structures, topology, vegetation and drivable paths. Stereo vision has been used extensively for this purpose. However, conventional single-baseline stereo does not scale well to different depths of perception. In this paper, a multi-baseline stereo frame is introduced to perform accurate 3D scene reconstruction from near range up to several meters away from the vehicle. A classifier that segments the scene into navigable and non-navigable areas based on 3D data is also described. It incorporates geometric features within an online self-learning framework to model and identify traversable ground, without any a priori assumption on the terrain characteristics. The ground model is automatically retrained during the robot motion, thus ensuring adaptation to environmental changes. The proposed strategy is of general applicability for robot’s perception and it can be implemented using any range sensor. Here, it is demonstrated for stereo-based data acquired by the multi-baseline device. Experimental tests, carried out in a rural environment with an off-road vehicle, are presented. It is shown that the use of a multi-baseline stereo frame allows for accurate reconstruction and scene segmentation at a wide range of visible distances, thus increasing the overall flexibility and reliability of the perception system.
Similar content being viewed by others
References
Hague T, Marchant JA, Tillett ND (2000) Ground-based sensing systems for autonomous agricultural vehicles. Comput Electron Agric 25(1–2):11–28
Rankin A, Huertas A, Matthies L (2005) Evaluation of stereo vision obstacle detecion algorithms for off-road autonomous navigation. In: Proceedings of the 32nd AUVSI symposium on unmanned systems, June 2005
Rovira-Más F, Zhang Q, Reid JF (2008) Stereo vision three-dimensional terrain maps for precision agriculture. Comput Electron Agric 60(2):133–143
Konolige K, Agrawal M, Bolles RC, Cowan C, Fischler M, Gerkey B (2008) Outdoor mapping and navigation using stereo vision. Exp Robot Springer Tracts Adv Robot 39:179–190
Milella A, Reina G, Siegwart R (2006) Computer vision methods for improved mobile robot state estimation in challenging terrains. J Multimed 1(7):49–61
Reina G, Milella A (2012) Towards autonomous agriculture: automatic ground detection using trinocular stereovision. Sensors 12(9):12405–12423
Okutomi M, Kanade T (1993) A multiple-baseline stereo. IEEE Trans Pattern Anal Mach Intell 15(4):353–363
Gallup D, Frahm JM, Mordohai P, Pollefeys M (2008) Variable baseline/resolution stereo. In: IEEE conference on computer vision and pattern recognition, Anchorage, AK, 23–28 June 2008, pp 1–8. doi:10.1109/CVPR.2008.4587671
Milella A, Reina G, Foglia M (2013) A multi-baseline stereo system for scene segmentation in natural environments. In: IEEE international conference on technologies for practical robot applications (TePRA), Woburn, MA, 22–23 Apr 2013, pp 1–6. doi:10.1109/TePRA.2013.6556370
Broggi A, Cappalunga A, Caraffi C, Cattani S, Ghidoni S, GrisleriP P, Porta P, Posterli M, Zani P (2010) TerraMax vision at the urban challenge 2007. IEEE Trans Intell Transp Syst 11(1):194–205
Olson CF, Abi-Rached H (2010) Wide-baseline stereo vision for terrain mapping. Mach Vis Appl 21:713–725
http://www.irstea.fr/en/institute. Accessed 26 Feb 2014
Broggi A, Caraffi C, Fedriga RI, Grisleri P (2005) Obstacle detection with stereo vision for off-road vehicle navigation. In: IEEE computer society conference on computer vision and pattern recognition, workshop, San Diego, CA, USA, 25 June 2005, p 65. doi:10.1109/CVPR.2005.503
Kelly A, Stentz A (1998) Stereo vision enhancements for low-cost outdoor autonomous vehicles. In: International conference on robotics and automation, workshop WS-7, navigation of outdoor autonomous vehicles (ICRA’98), May 1998
Manduchi R, Castano A, Talukder A, Matthies L (2003) Obstacle detection and terrain classification for autonomous off-road navigation. Auton Robots 18:81–102
Moravec A (1981) Rover visual obstacle avoidance. In: Proceedings of the 7th international joint conference on artificial intelligence, Vancouver, British Columbia, pp 785–790
Klarquist W, Bovik A (1997) Adaptive variable baseline stereo for vergence control. In: Proceedings of the 1997 IEEE international conference on robotics and automation, vol. 3, pp 1952–1959
Nakabo Y, Mukai T, Hattori Y, Takeuchi Y, Ohnishi N (2005) Variable baseline stereo tracking vision system using high-speed linear slider. In: Proceedings of the 2005 IEEE international conference on robotics and automation, pp 1567–1572
Milella A, Reina G, Underwood J, Douillard B (2011) Combining radar and vision for self-supervised ground segmentation in outdoor environments. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 255–260
Milella A, Reina G, Underwood J, Douillard B (2014) Visual ground segmentation by radar supervision. Robot Auton Syst doi:10.1016/j.robot.2012.10.001 (in press)
Milella A, Reina G, Underwood J (2014) A self-learning framework for statistical ground classification using radar and monocular vision. J Field Robot (in press)
Stavens D, Thrun S (2006) A self-supervised terrain roughness estimator for offroad autonomous driving. In: Proceedings of the conference on uncertainty in AI (UAI), pp 13–16
Zhou S, Xi J, McDaniel MW, Nishihata T, Salesses P, Iagnemma K (2012) Self-supervised learning to visually detect terrain surfaces for autonomous robots operating in forested terrain. J Field Robot 29(2):277–297
Reina G, Milella A, Underwood J (2012) Self-learning classification of radar feautures for scene understanding. Robot Auton Syst 60(11):1377–1388
Vernaza P, Taskar B, Lee DD (2008) Online, self-supervised terrain classification via discriminatively trained submodular Markov random fields. In: Proceedings of IEEE international conference on robotics and automation, pp 2750–2757
Hadsell R, Sermanet P, Ben J, Erkan A, Scoffier M, Kavukcuoglu K, Muller U, LeCun Y (2009) Learning long-range vision for autonomous off-road driving. J Field Robot 26(2):120–144
Konolige K, Agrawal M, Blas MR, Bolles RC, Gerkey BP, Solá J, Sundaresan A (2009) Mapping, navigation, and learning for off-road traversal. J Field Robot 26(1):88–113
Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media, USA
Kuthirummal S, Das A, Samarasekera S (2011) A graph traversal based algorithm for obstacle detection using lidar or stereo. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), San Francisco, CA, 25–30 Sept 2011, pp 3874–3880. doi:10.1109/IROS.2011.6094685
Duda EO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley, New York
Mardia K, Kent J, Bibby J (1979) Multivariate analysis. Academic Press, London
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Milella, A., Reina, G. 3D reconstruction and classification of natural environments by an autonomous vehicle using multi-baseline stereo. Intel Serv Robotics 7, 79–92 (2014). https://doi.org/10.1007/s11370-014-0146-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11370-014-0146-x